Skip to content

branch-3.0: [bug](cloud) Fix npe when show cluster load statistic #50866 #50992

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
dataroaring merged 1 commit into from
May 17, 2025
Merged

branch-3.0: [bug](cloud) Fix npe when show cluster load statistic #50866 #50992

dataroaring merged 1 commit into from
May 17, 2025

Conversation

@github-actions
Copy link
Contributor

@github-actions github-actions bot commented May 17, 2025

Cherry-picked from #50866

@xy720
[bug](cloud) Fix npe when show cluster load statistic (#50866)
16d3d69 
### What problem does this PR solve?

In cloud mode:

before
```
MySQL [(none)]> SHOW PROC '/cluster_balance/cluster_load_stat/location_default/REMOTE_CACHE';
ERROR 1105 (HY000): errCode = 2, detailMessage = Cannot invoke "org.apache.doris.clone.LoadStatisticForTag.getStatistic(org.apache.doris.thrift.TStorageMedium)" because "loadStatisticForTag" is null
```

after
```
MySQL [(none)]> SHOW PROC "/cluster_balance/cluster_load_stat/location_default/REMOTE_CACHE";
Empty set (0.00 sec)
```

And may be we should support show cluster load stat by compute-group
later.
@github-actions github-actions bot requested a review from dataroaring as a code owner May 17, 2025 03:47
@hello-stephen
Copy link
Contributor

hello-stephen commented May 17, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@dataroaring dataroaring reopened this May 17, 2025
@hello-stephen
Copy link
Contributor

hello-stephen commented May 17, 2025

run buildall

@doris-robot
Copy link

doris-robot commented May 17, 2025

TPC-H: Total hot run time: 39868 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 16d3d69dff545eb07e5a5d38b01b610e4b19591b, data reload: false

------ Round 1 ----------------------------------
q1	17578	7474	6624	6624
q2	2085	179	163	163
q3	10570	1099	1118	1099
q4	10579	716	747	716
q5	7757	2901	2736	2736
q6	221	135	132	132
q7	967	614	604	604
q8	9360	1985	2062	1985
q9	6647	6401	6423	6401
q10	6993	2263	2272	2263
q11	476	254	256	254
q12	397	206	200	200
q13	17799	2952	2970	2952
q14	236	207	201	201
q15	507	455	455	455
q16	662	591	572	572
q17	993	573	524	524
q18	7339	6597	6567	6567
q19	1407	1052	1064	1052
q20	503	208	205	205
q21	4096	3187	3158	3158
q22	1113	1015	1005	1005
Total cold run time: 108285 ms
Total hot run time: 39868 ms

----- Round 2, with runtime_filter_mode=off -----
q1	6584	6572	6554	6554
q2	326	237	233	233
q3	2966	2775	2906	2775
q4	2035	1775	1775	1775
q5	5740	5751	5792	5751
q6	210	134	131	131
q7	2244	1804	1831	1804
q8	3362	3548	3539	3539
q9	8992	8821	8865	8821
q10	3529	3525	3533	3525
q11	582	497	508	497
q12	814	579	576	576
q13	9567	3127	3190	3127
q14	302	258	272	258
q15	523	479	461	461
q16	685	630	662	630
q17	1861	1602	1619	1602
q18	8137	7693	7796	7693
q19	1695	1645	1562	1562
q20	2039	1832	1842	1832
q21	5502	5304	5312	5304
q22	1104	1070	1031	1031
Total cold run time: 68799 ms
Total hot run time: 59481 ms

@doris-robot
Copy link

doris-robot commented May 17, 2025

TPC-DS: Total hot run time: 197270 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 16d3d69dff545eb07e5a5d38b01b610e4b19591b, data reload: false

query1	1310	919	884	884
query2	6266	2017	1956	1956
query3	10935	4275	4275	4275
query4	61458	29646	23725	23725
query5	5188	459	451	451
query6	402	171	203	171
query7	5471	304	310	304
query8	320	217	219	217
query9	8662	2593	2589	2589
query10	451	271	262	262
query11	17735	15156	15901	15156
query12	158	102	109	102
query13	1424	426	436	426
query14	10509	7429	7291	7291
query15	208	190	189	189
query16	7083	491	509	491
query17	1161	587	592	587
query18	1820	328	318	318
query19	212	166	165	165
query20	125	111	112	111
query21	213	108	102	102
query22	4626	4386	4819	4386
query23	34201	34209	33967	33967
query24	6135	2947	2914	2914
query25	520	399	383	383
query26	651	162	163	162
query27	1745	359	361	359
query28	3957	2475	2430	2430
query29	688	443	444	443
query30	244	162	158	158
query31	1001	801	862	801
query32	65	53	53	53
query33	458	283	295	283
query34	937	505	519	505
query35	860	732	718	718
query36	1075	974	946	946
query37	117	68	65	65
query38	3974	3997	4007	3997
query39	1508	1462	1460	1460
query40	208	100	106	100
query41	51	50	46	46
query42	114	101	95	95
query43	535	494	490	490
query44	1175	811	832	811
query45	187	169	169	169
query46	1163	723	715	715
query47	1998	1915	1930	1915
query48	486	383	386	383
query49	727	398	391	391
query50	855	434	419	419
query51	7367	7374	7218	7218
query52	101	91	94	91
query53	278	186	187	186
query54	595	484	464	464
query55	87	77	76	76
query56	253	231	266	231
query57	1248	1175	1142	1142
query58	216	211	207	207
query59	3123	2930	2964	2930
query60	288	266	272	266
query61	122	118	111	111
query62	798	710	691	691
query63	222	200	188	188
query64	1395	673	649	649
query65	3296	3189	3186	3186
query66	699	301	296	296
query67	15891	15623	15378	15378
query68	4087	580	567	567
query69	419	258	261	258
query70	1208	1090	1098	1090
query71	349	251	249	249
query72	6432	4014	3932	3932
query73	749	341	344	341
query74	9993	9261	9308	9261
query75	3355	2662	2647	2647
query76	1937	1132	1033	1033
query77	505	281	272	272
query78	10483	9586	9582	9582
query79	1412	589	587	587
query80	838	430	423	423
query81	509	236	241	236
query82	1222	91	87	87
query83	163	146	142	142
query84	282	80	76	76
query85	860	301	291	291
query86	330	280	303	280
query87	4366	4222	4214	4214
query88	3961	2364	2340	2340
query89	419	292	290	290
query90	2035	185	185	185
query91	182	146	146	146
query92	66	48	51	48
query93	1884	547	562	547
query94	739	302	304	302
query95	354	259	254	254
query96	612	282	287	282
query97	3349	3167	3170	3167
query98	218	201	197	197
query99	1541	1316	1298	1298
Total cold run time: 312210 ms
Total hot run time: 197270 ms

@doris-robot
Copy link

doris-robot commented May 17, 2025

ClickBench: Total hot run time: 32.96 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 16d3d69dff545eb07e5a5d38b01b610e4b19591b, data reload: false

query1	0.03	0.03	0.03
query2	0.07	0.03	0.03
query3	0.23	0.07	0.06
query4	1.63	0.10	0.10
query5	0.52	0.50	0.49
query6	1.14	0.74	0.73
query7	0.02	0.02	0.02
query8	0.03	0.03	0.03
query9	0.56	0.50	0.50
query10	0.55	0.55	0.58
query11	0.14	0.11	0.11
query12	0.13	0.10	0.11
query13	0.61	0.59	0.60
query14	2.76	2.74	2.68
query15	0.89	0.81	0.83
query16	0.41	0.38	0.37
query17	1.03	1.06	1.01
query18	0.24	0.21	0.21
query19	1.88	1.84	2.01
query20	0.01	0.01	0.02
query21	15.38	0.59	0.59
query22	2.55	2.13	1.93
query23	16.89	0.82	0.83
query24	3.40	1.75	1.71
query25	0.23	0.11	0.06
query26	0.48	0.14	0.14
query27	0.04	0.05	0.04
query28	8.91	0.54	0.49
query29	12.62	3.29	3.20
query30	0.24	0.06	0.06
query31	2.87	0.38	0.38
query32	3.25	0.47	0.46
query33	2.98	2.95	2.99
query34	17.14	4.53	4.51
query35	4.53	4.56	4.56
query36	0.68	0.48	0.49
query37	0.08	0.06	0.06
query38	0.04	0.04	0.04
query39	0.03	0.03	0.02
query40	0.16	0.13	0.12
query41	0.07	0.02	0.02
query42	0.04	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 105.53 s
Total hot run time: 32.96 s

dataroaring
dataroaring approved these changes May 17, 2025
View reviewed changes
Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@dataroaring dataroaring merged commit 25d7592 into branch-3.0 May 17, 2025
23 of 24 checks passed
@github-actions github-actions bot deleted the auto-pick-50866-branch-3.0 branch May 17, 2025 13:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@dataroaring dataroaring dataroaring approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@hello-stephen @doris-robot @dataroaring @xy720